Scalable video decoder and controlling method for the same

ABSTRACT

A scalable video decoder and a controlling method thereof are provided. A method of controlling a scalable video decoder includes: determining whether a macro block mode of an upper layer is an interlayer prediction mode when decoding a scalable video; when the macro block mode of the upper layer is determined as the interlayer prediction mode, determining an up-sampling operation is required to thereby either perform texture up-sampling using corresponding macro block information of a lower layer and neighboring pixel information, or perform residual up-sampling using the corresponding macro block information of the lower layer; and decoding the upper layer using the texture up-sampling or the residual up-sampling.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority from Korean Patent Application No.10-2008-0071563, filed on Jul. 23, 2008, in the Korean IntellectualProperty Office, the entire disclosure of which is incorporated hereinby reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a scalable video decoder and acontrolling method thereof, and more particularly, to a scalable videodecoder that can reduce the usage amount of a memory when up-sampling avideo block in scalable decoding of an image signal, and a controllingmethod thereof.

2. Description of Related Art

A scalable video codec (SVC) scheme denotes a scheme that encodes animage signal to the best quality. Also, although a partial sequence ofresultantly generated picture sequence, that is, a sequence of a framethat is partially selected from all sequences, is decoded and used, theSVC scheme enables low quality video expression.

A hierarchical B picture (H-B picture) scheme is an encoding scheme thatis proposed to be used for the above SVC.

The picture sequence, encoded using the H-B picture corresponding to theSVC scheme, receives and processes only the partial sequence to therebyenable the low quality video expression. However, when a bitrate isdecreased, the quality may be even more degraded. In order to solve thisproblem, it is possible to hierarchically provide separate auxiliarypicture sequences with a low transmission bitrate, for example, picturesequences with a low frame rate per second.

Also, up-sampling or down-sampling of an image may be required in orderto simultaneously transmit a large image and a small image.Specifically, it is possible to encode a single image signal source intoeach of a four common intermediate format (CIF) picture sequence, a CIFpicture sequence, and a quarter common intermediate format (QCIF)picture sequence, and transmit the encoded picture sequences to adecoding apparatus.

Since both an upper layer and a lower layer encode the same image signalsource, redundancy exists in the encoded signal of both the upper layerand the lower layer.

Therefore, in order to increase a coding rate of a particular layer thatis encoded according to a scheme of transmitting various types of screensizes, an image frame of a current layer at the same point in time as animage frame of a lower layer is created as a predicted image, that is,residual data based on the image frame of the lower layer. For example,when encoding a current macro bloc of a high resolution image into anintra BL mode, a corresponding macro block of the lower layer, that is,a block including a region that corresponds to the same time as thecurrent macro block and also corresponds to the same location in a frameis up-sampled and then a difference value between pixel values of theup-sampled block or an error value is encoded in the current macroblock.

The up-sampled block is not transmitted to a decoder. Thus, so that thedecoder may decode the encoded macro block, the decoder must up-sample acorresponding macro block of the lower layer and use the up-sampledmacro block. The corresponding macro block of the lower block needs tobe up-sampled not only for encoding the macro block of the intra BLmode, but also for performing a residual data prediction operationbetween layers.

As described above, when providing a plurality of layers with adifferent picture size or resolution from encoding streams, there is aneed for up-sampling an image block during an encoding and decodingprocess.

SUMMARY OF THE INVENTION

An aspect of the present invention provides a scalable video decoder anda controlling method thereof that can inspect a mode of a current macroblock and determine whether it is required to enlarge, that is,up-sample an image when decoding a small size screen (lower layer) andthen decoding a large size screen (upper layer) using characteristics ofa single-loop-decoding mode of H.264 scalable video codec (SVC), and canperform an image up-sampling operation using macro block information ofa lower layer and only a portion of neighboring pixels when textureup-sampling or residual up-sampling is required, and thereby reducing arequired amount of memory used.

According to an aspect of the present invention, there is provided amethod of controlling a scalable video decoder, the method including:determining whether a macro block mode of an upper layer is aninterlayer prediction mode when decoding a scalable video; when themacro block mode of the upper layer is determined as the interlayerprediction mode, determining an up-sampling operation is required tothereby either perform texture up-sampling using corresponding macroblock information of a lower layer and neighboring pixel information, orperform residual up-sampling using the corresponding macro blockinformation of the lower layer; and decoding the upper layer using thetexture up-sampling or the residual up-sampling.

In this instance, the neighboring pixel information may includeinformation associated with four pixels adjacent to the correspondingmacro block.

Also, the performing of the texture up-sampling or the residualup-sampling may include performing texture up-sampling by copying thecorresponding macro block of the lower layer and two pixels adjacent tothe corresponding macro block in a 20×20 memory and using a 4-taptexture up-sampling filter.

Also, the performing of the texture up-sampling or the residualup-sampling may include performing residual up-sampling by copying aresidual value of the corresponding macro block of the lower layer in a16×16 memory and using a 2-tap residual up-sampling filter.

Also, the performing of the texture up-sampling or the residualup-sampling may include performing image up-sampling by performinghorizontal convolution and then performing vertical convolution, or byperforming vertical convolution and then performing horizontalconvolution.

Also, the upper layer may be two times as wide and two times as long asthe lower layer.

According to another aspect of the present invention, there is provideda scalable video decoder including: a macro block mode decision unit todetermine whether a macro block mode of an upper layer is an interlayerprediction mode when decoding a scalable video; an up-sampling unit todetermine, when the macro block mode of the upper layer is determined asthe interlayer prediction mode, an up-sampling operation is required tothereby either perform texture up-sampling using informationcorresponding macro block information of a lower layer and neighboringpixel information, or to perform residual up-sampling using thecorresponding macro block information of the lower layer; and a decodingunit to decode the upper layer using the texture up-sampling or theresidual up-sampling.

ADVANTAGEOUS EFFECT

According to the present invention, when decoding a small size screen(lower layer) and then decoding a large size screen (upper layer) usingcharacteristics of a single-loop-decoding mode of H.264 scalable videocodec (SVC), it is possible to inspect a mode of a current macro blockand determine whether it is required to enlarge, that is, up-sample animage. As a result of the determination, when texture up-sampling orresidual up-sampling is required, it is possible to perform an imageup-sampling operation using macro block information of a lower layer andonly a portion of neighboring pixels. Through this, it is possible toreduce a required amount of memory used.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects of the present invention will becomeapparent and more readily appreciated from the following detaileddescription of certain exemplary embodiments of the invention, taken inconjunction with the accompanying drawings of which:

FIG. 1 is a block diagram illustrating a configuration of a scalablevideo decoder according to an embodiment of the present invention;

FIG. 2 illustrates a scheme of up-sampling only a block coded using anintra LB mode among macro blocks of a lower layer according to anembodiment of the present invention;

FIG. 3 illustrates a scheme of up-sampling an image according to anembodiment of the present invention;

FIG. 4 illustrates a scheme of performing texture up-sampling usingcorresponding macro block information of a lower layer and neighboringpixel information according to an embodiment of the present invention;

FIG. 5 illustrates a scheme of performing residual up-sampling usingcorresponding macro block information of a lower layer according to anembodiment of the present invention; and

FIG. 6 is a flowchart illustrating a method of controlling a scalablevideo decoder according to an embodiment of the present invention.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

Reference will now be made in detail to exemplary embodiments of thepresent invention, examples of which are illustrated in the accompanyingdrawings, wherein like reference numerals refer to the like elementsthroughout. The exemplary embodiments are described below in order toexplain the present invention by referring to the figures.

FIG. 1 is a block diagram illustrating a configuration of a scalablevideo decoder according to an embodiment of the present invention. Thescalable video decoder includes a block mode decision unit 110, anup-sampling unit 120, and a decoding unit 130.

The macro block mode decision unit 110 may determine whether a macroblock mode of an upper layer is an interlayer prediction mode whendecoding a scalable video.

When the macro block mode of the upper layer is determined as theinterlayer prediction mode, the up-sampling unit 120 may determine anup-sampling operation is required to thereby perform the up-samplingoperation.

In this instance, the up-sampling unit 120 may perform textureup-sampling using corresponding macro block information of a lower layerand neighboring pixel information, or may perform residual up-samplingusing the corresponding macro block information of the lower layer.

In particular, when the up-sampling unit 120 performs textureup-sampling, the up-sampling unit 120 may perform up-sampling based oninformation associated with four pixels adjacent to the correspondingmacro block. In this instance, the up-sampling unit 120 may performtexture up-sampling by copying the corresponding macro block of thelower layer and two pixels adjacent to the corresponding macro block ina 20×20 memory and using a 4-tap texture up-sampling filter.

Also, the up-sampling unit 120 may perform residual up-sampling bycopying a residual value of the corresponding macro block of the lowerlayer in a 16×16 memory and using a 2-tap residual up-sampling filter.

Also, the up-sampling unit 120 may perform image up-sampling byperforming horizontal convolution and then performing verticalconvolution, or by performing vertical convolution and then performinghorizontal convolution.

The above process may be applicable when the upper layer is two times aswide and two times as long as the lower layer.

The decoding unit 130 may decode the upper layer using the textureup-sampling or the residual up-sampling.

Therefore, when decoding the upper layer, it is possible to reduce arequired usage amount of a memory through a process of inspecting acurrent macro block mode, determining whether up-sampling of an image isrequired, and when texture up-sampling or residual up-sampling isrequired as a result of the determination, performing an imageup-sampling operation using macro block information of a lower layer andonly a portion of neighboring pixels.

FIG. 2 illustrates a scheme of performing an up-sampling operation foronly a block coded using an intra BL mode among macro blocks of a lowerlayer according to an embodiment of the present invention. Hereinafter,the above scheme will be described with reference to FIG. 2.

The up-sampling operation of the image is an essential operation for anSVC encoder/decoder. H.264 SVC, currently in a standardization process,adopts a single-loop-decoding scheme. The single-loop-decoding schemedenotes a scheme that performs the up-sampling operation for only theintra BL mode coded block among macro blocks of a lower layer 210 withthe small size screen to thereby use as a prediction signal forreconstruction of the intra mode of a current layer. Accordingly, theencoder/decoder may effectively perform the image up-sampling operationusing characteristics of a single-loop-decoding mode.

Also, coding modes I_(—)16×16, I_(—)4×4, P_(—)16×16, P_(—)8×8, etc., ofexisting H.264 and encoding modes of SVC base_mode_flag,residual_prediction_flag, etc. may be simultaneously applicable to anupper layer 220. Therefore, when decoding the upper layer 220, it ispossible to inspect an encoding mode and determine whether to reuseinformation of the lower layer 210. Through this, it is possible toreduce a required memory capacity.

FIG. 3 illustrates a scheme of up-sampling an image according to anembodiment of the present invention. The image up-sampling scheme willbe described with reference to FIG. 3.

As shown in FIG. 3, a horizontal and vertical convolution process usinga filter may be performed using a base layer image 310 corresponding toa restored image of a lower layer.

A temporary image 320 corresponding to a horizontally up-sampled image,that is, enlarged image may be created by performing horizontalconvolution for the base layer image 310. An up-sampled image 330 may becreated by performing vertical convolution for the temporary image 320.

In this instance, although an order of the horizontal convolution andthe vertical convolution is changed, the same result may be obtained.

Specifically, up-sampling may be performed by performing, via a filter,horizontal convolution and vertical convolution for the base layer image310 corresponding to the restored image of the lower layer. Inparticular, up-sampling may be performed by initially performing thehorizontal convolution and then performing the vertical convolution.Conversely, up-sampling may be performed by performing the verticalconvolution and then performing the horizontal convolution.

When performing up-sampling for the lower layer, up-samplingcorresponding to enlargement may be performed by setting a predeterminedmultiple of the lower layer as a ratio.

FIG. 4 illustrates a scheme of performing texture up-sampling usingcorresponding macro block information of a lower layer and neighboringpixel information according to an embodiment of the present invention.Hereinafter, the texture up-sampling scheme will be described withreference to FIG. 4.

When an interlayer prediction mode of an upper layer requires thetexture up-sampling, the texture up-sampling may be performed based oninformation associated with a corresponding base layer macro block 410corresponding to a corresponding macro block of a lower layer and pixels420 adjacent to the corresponding macro block.

More specifically, texture up-sampling may be performed by copying thecorresponding macro block of the lower layer, that is, the correspondingbase layer corresponding macro block, and the adjacent two pixels 420 ina 20×20 memory and using a 4-tap texture up-sampling filter.

In this instance, when performing two-dimensional up-sampling in adecoder, an order of vertical convolution and horizontal convolution maybe randomly performed. Therefore, a 20×20 memory may be required.

FIG. 5 illustrates a scheme of performing residual up-sampling usingcorresponding macro block information of a lower layer according to anembodiment of the present invention. Hereinafter, the residualup-sampling scheme will be described with reference to FIG. 5.

When an interlayer prediction mode of an upper layer requires theresidual up-sampling, the residual up-sampling may be performed based oninformation associated with a corresponding base layer macro block 510corresponding to a corresponding macro block of a lower layer.

More specifically, residual up-sampling may be performed by copying aresidual value of the corresponding macro block 510 of the lower layer,that is, the corresponding base layer of the corresponding macro block,in a 16×16 memory and using a 2-tap residual up-sampling filter. In thisinstance, an appropriate padding operation may be performed in a blockboundary according to H.264 SVC standard.

Unlike the existing scheme that requires texture and residualup-sampling information corresponding to the image size of the upperlayer, the present invention shown in FIGS. 4 and 5 requires onlyadditional memories, a 40×40 memory and a 32×32 memory that are two foldof the 20×20 memory and the 16×16 memory. In particular, in the case ofhigh quality decoding of a high definition (HD) class, a required usageamount of memory may be significantly reduced in comparison to theexisting scheme.

FIG. 6 is a flowchart illustrating a method of controlling a scalablevideo decoder according to an embodiment of the present invention.Hereinafter, the method of controlling the scalable video decoder willbe described with reference to FIG. 6.

In operation S610, when decoding a scalable video, the method maydetermine whether a macro block mode of an upper layer is an interlayerprediction mode.

In operation S620, when the macro block mode of the upper layer isdetermined as the interlayer prediction mode, the method may determinean up-sampling operation is required to thereby perform the up-samplingoperation.

In this instance, the method may perform texture up-sampling usingcorresponding macro block information of a lower layer and neighboringpixel information, or may perform residual up-sampling using thecorresponding macro block information of the lower layer.

In particular, when the method performs texture up-sampling, the methodmay perform up-sampling based on information associated with four pixelsadjacent to the corresponding macro block. In this instance, the methodmay perform texture up-sampling by copying the corresponding macro blockof the lower layer and two pixels adjacent to the corresponding macroblock in a 20×20 memory and using a 4-tap texture up-sampling filter.

Also, the method may perform residual up-sampling by copying a residualvalue of the corresponding macro block of the lower layer in a 16×16memory and using a 2-tap residual up-sampling filter.

Also, the method may perform image up-sampling by performing horizontalconvolution and then performing vertical convolution, or by performingvertical convolution and then performing horizontal convolution. Theabove process may be applicable when the upper layer is two times aswide and two times as long as the lower layer.

In operation S630, the method may decode the upper layer using thetexture up-sampling or the residual up-sampling.

Conversely, when the macro block mode of the upper layer is not theinterlay prediction mode, the method may decode the upper layer withoutapplying a separate up-sampling operation.

Therefore, when decoding the upper layer, it is possible to reduce arequired usage amount of a memory through a process of inspecting acurrent macro block mode, determining whether up-sampling of an image isrequired, and when texture up-sampling or residual up-sampling isrequired as a result of the determination, performing an imageup-sampling operation using macro block information of a lower layer andonly a portion of neighboring pixels.

Although a few exemplary embodiments of the present invention have beenshown and described, the present invention is not limited to thedescribed exemplary embodiments. Instead, it would be appreciated bythose skilled in the art that changes may be made to these exemplaryembodiments without departing from the principles and spirit of theinvention, the scope of which is defined by the claims and theirequivalents.

1. A method of controlling a scalable video decoder, the methodcomprising: determining whether a macro block mode of an upper layer isan interlayer prediction mode when decoding a scalable video; when themacro block mode of the upper layer is determined as the interlayerprediction mode, determining an up-sampling operation is required tothereby either perform texture up-sampling using corresponding macroblock information of a lower layer and neighboring pixel information, orperform residual up-sampling using the corresponding macro blockinformation of the lower layer; and decoding the upper layer using thetexture up-sampling or the residual up-sampling.
 2. The method of claim1, wherein the neighboring pixel information comprises informationassociated with four pixels adjacent to the corresponding macro block.3. The method of claim 1, wherein the performing of the textureup-sampling or the residual up-sampling comprises performing textureup-sampling by copying the corresponding macro block of the lower layerand two pixels adjacent to the corresponding macro block in a 20×20memory and using a 4-tap texture up-sampling filter.
 4. The method ofclaim 1, wherein the performing of the texture up-sampling or theresidual up-sampling comprises performing residual up-sampling bycopying a residual value of the corresponding macro block of the lowerlayer in a 16×16 memory and using a 2-tap residual up-sampling filter.5. The method of claim 1, wherein the performing of the textureup-sampling or the residual up-sampling comprises performing imageup-sampling by performing horizontal convolution and then performingvertical convolution, or by performing vertical convolution and thenperforming horizontal convolution.
 6. The method of claim 1, wherein theupper layer is two times as wide and two times as long as the lowerlayer.
 7. A scalable video decoder comprising: a macro block modedecision unit to determine whether a macro block mode of an upper layeris an interlayer prediction mode when decoding a scalable video; anup-sampling unit to determine, when the macro block mode of the upperlayer is determined as the interlayer prediction mode, an up-samplingoperation is required to thereby either perform texture up-samplingusing information corresponding macro block information of a lower layerand neighboring pixel information, or to perform residual up-samplingusing the corresponding macro block information of the lower layer; anda decoding unit to decode the upper layer using the texture up-samplingor the residual up-sampling.
 8. The decoder of claim 7, wherein theneighboring pixel information comprises information associated with fourpixels adjacent to the corresponding macro block.
 9. The decoder ofclaim 7, wherein the up-sampling unit performs texture up-sampling bycopying the corresponding macro block of the lower layer and two pixelsadjacent to the corresponding macro block in a 20×20 memory and using a4-tap texture up-sampling filter.
 10. The decoder of claim 7, whereinthe up-sampling unit performs residual up-sampling by copying a residualvalue of the corresponding macro block of the lower layer in a 16×16memory and using a 2-tap residual up-sampling filter.
 11. The decoder ofclaim 7, wherein the up-sampling unit performs image up-sampling byperforming horizontal convolution and then performing verticalconvolution, or by performing vertical convolution and then performinghorizontal convolution.
 12. The decoder of claim 7, wherein the upperlayer is two times as wide and two times as long as the lower layer.